3D Face Reconstruction and Gaze Tracking in the HMD for Virtual Interaction `

With the rapid development of virtual reality (VR) technology, VR headsets, a.k.a. Head-Mounted Displays (HMDs), are widely available, allowing immersive 3D content to be viewed. A natural need for truly is allow bidirectional communication: user should able interact with world using facial expressions and eye gaze, in addition traditional means interaction. The typical application scenario includes conferencing roaming, where ideally users see other have contact them world. In addition, gaze also provides a interaction objects. Despite significant achievements recent years reconstruction faces from RGB or RGB-D images, it remains challenge reliably capture reconstruct including when wearing an HMD, because majority face occluded, especially those areas around eyes which essential recognizing gaze. this paper, we introduce novel real-time system that HMDs, robustly recover We further propose method map directions world, useful interactive mode VR. compare our state of-the-art techniques both qualitatively quantitatively, demonstrate effectiveness live capture.